Papers with multimedia event extraction
Joint Multimedia Event Extraction from Video and Article (2021.findings-emnlp)
Copied to clipboard
Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, Shih-Fu Chang
| Challenge: | Existing methods to extract multimedia events from video and text are limited to video and images. |
| Approach: | They propose a task to jointly extract events from video and text documents . they propose 'self-supervised' cross-modal event coreference model and cross-mod transformer architecture . |
| Outcome: | The proposed method achieves 6.0% and 5.8% absolute F-score gain on video-article pairs . the proposed method can resolve coreference and extract multimodal event frames more accurately than existing methods. |
Cross-media Structured Common Space for Multimedia Event Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | We propose a new task to extract events and their arguments from multimedia documents . traditional methods target text, images or videos, but multimedia content is distributed via multimedia . |
| Approach: | They propose a method that encodes structured representations of semantic information from textual and visual data into a common embedding space. |
| Outcome: | The proposed method achieves 4.0% and 9.8% absolute gains on text event argument role labeling and visual event extraction. |
Evaluation Pitfalls and Challenges in Multimedia Event Extraction (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has focused on textual content, but recent work has explored the integration of additional modalities to support more accurate and comprehensive event understanding. |
| Approach: | They propose to analyze the evaluation pitfalls of multimedia event extraction by combining textual and visual inputs to identify events and their arguments across multiple modalities. |
| Outcome: | The proposed model overestimates performance and performance of the proposed model in a series of controlled experiments under a strict evaluation framework. |